This invention relates generally to user interfaces to computational devices, and more particularly to applications in which displayed objects are selected using a mouse or stylus. Two types of interactive drawing/sketching/editing applications are currently in use, both of which support creation of new image material, through draw operations, and selection and manipulation of existing material, through editing operations. The types of interactive applications are distinguished by the emphasis placed on “sketch” and “editing” operations. In an image “editing” program, selection and manipulation of image objects is the primary activity. Therefore, stylus or mouse interaction is designed primarily to interpret stylus input as selection gestures, and the default interpretation of mouse or stylus activity is selection of existing image objects for manipulation. Tools for drawing objects are provided by auxiliary command objects, usually menus.
In a “sketch” program, however, the primary activity is the “draw” operation. To facilitate the sketching process, it is important for users to be able to quickly execute a series of markings such as handwritten or sketched strokes, without having to perform a menu initiation command at every stroke. These programs are designed such that draw operations can be the default interpretation of mouse or stylus activity. The disadvantage to this type of program is that when priority is placed on draw operations, selection operations become demoted and require explicit menu choices or button clicks to invoke a selection, which impedes the smooth flow of multiple selection and manipulation operations.
Using these currently available tools, it can be very difficult to select and move, delete, or otherwise modify salient collections of markings at will. In particular, many user interfaces offer object selection by using the mouse to drag a rectangle enclosing target objects, draw a freeform path enclosing target objects, or define the vertices of a polygon enclosing target objects. There are no extant techniques for making all three of these selection methods available without the user specifying manually which one they intend to use. Additionally, the step of choosing among selection modes disrupts the flow of the task.
U.S. Pat. No. 5,485,565 to Saund et al. titled “Gestural Indicators for Selecting Graphic Objects” discloses a graphical imaging system, in which the rough location, size and shape of objects in the image is summarized by a first characteristic descriptor, representing a parametric “pose” computed for each object. A second characteristic descriptor, for example a “gesture matching” function, is provided in order to select the single object, or the set of objects, that best comports with the user's selection gesture. When most closely matched, these key characteristic descriptors permit simple and natural user gestures to distinguish among a large set of graphic objects that may overlap both spatially. User gestures can be simple slashes passing through the object, or quick, coarse approximations of objects' shapes.
U.S. Pat. No. 5,513,309 to Meier et al. titled “Graphic Editor User Interface for a Pointer-Based Computer System” discloses a graphical editor arranged to permit the user to edit selected graphic objects by highlighting the objects and moving them with editing handles. A bounding box is also drawn about the selected portions of the object. In various aspects of the invention, the user is permitted to edit the object by executing specific actions, including resizing, duplicating, distorting and moving either the entire object or only selected portions. After any of the editing operations is performed, the display is updated to reflect changes made during the editing step.
U.S. Pat. No. 5,523,775 (1996) to Capps titled “Method for Selecting Objects on a Computer Display” teaches a method for selecting objects on a screen of a pen-based computer system. The user's intention to enter the selection mode is indicated by the user's performing a special action with the stylus, such as holding the stylus in one place for a minimum amount of time, tapping the stylus immediately prior to and in the same location the highlighted gesture is to begin, or drawing a small circle. Each of these approaches is prone to user error. For example, a user may place the stylus down while thinking about the shape of the stroke to be drawn and unexpectedly be placed in a “highlight” (selection) mode. Similar problems occur with the “tap-and-a-half” and “small circle” methods described in Capps. Like other existing methods in this field, under the Capps invention the user is required to declare in advance of their stroke that the stroke is to be interpreted in terms of “highlight” (select) mode.
U.S. Pat. No. 5,760,773 to Berman et al. titled “Methods and Apparatus for Interacting with Data Objects Using Action Handles” teaches a central processing unit coupled to a pointer control device such as a pen, stylus or mouse, that permits the user to selectively position a pointer and activate an action handle on a display associated with a data object. Activation of the action handle signals the central processing unit of selections associated with the data object. Tapping or clicking on the action handle causes display of a context menu containing at least one command that may be invoked with respect to the data object. Dragging the action handle indicates movement or dragging of the action handle and the corresponding data object for an operation associated with dragging such as drag-and-drop.
The present invention offers a user interface technique permitting users to perform any of four types of selection method (click/tap, rectangle drag, path draw, and polygon draw) using a mouse or stylus without having to specify among these selection methods explicitly using a menu or other intermediate step.